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Backgr^nd of Project 



A cross-section of Texas citizenry has identified a number of 

characteristics considered basic for young adults to possess in 

» * » 

order to make valid career « decisions . These chatacteJristics 

« * 

termed "learner outcomes 11 range from fundamental knowledge and 

A} ' 

skills required for all jobs to the development of sophisticated 
value systems about careet options, and the work .ethic. In order ' 
to determine the extent to Vhich individual students and groups 
of students have mastered thesa bapic learner outcomes, a coopera- 
tive effort is underway to develop sets of measurement instruments, 

\ 

Developmental Steps - ' 

Items have been written as potential measurejsf o\ ^78 outcomes in* 
the following nine categories: 



I. Career Planning and Decision Making . < \ N 

II. Career Information ■ * 
III* Job Acquisition and Retention 

IV. Attitudes and Appreciation for a Career Success 

V. Skills in Human^ Relationships for Carpers 

VI. Self -Investigation and Evaluation for Career Success 

VII. v Person/Work/Societal Responsibilities . 
.VIII. ' Economic Factors Influencing Chreer Opportunity 
IX. \ JSducation/Career Opportunity Relationships 

7 .*\ • 

The ite^jts have fceen arranged into two levels of pil)^ instruments 

designec^for students in grades seven through eleven] In March and 

April of this ,year twenty- two instruments were tested on a random 

sample of s^me 524 cla'ssrooms ^n4^15,000 students throughout the. 

State of Texas. The purpose of this pilot testing was to determine 

+ f 



/ 



if the items were adequate measures of outcomes, to see if the 
-directions for administration were understandable by teachers and 
students, and to find how best to score the instruments and report, 
results to students and teachers. 

The Evaluation Questionnaire 

■ . - ' ■ \ 

In order to obtain information about the, usefulness of the Career 
Education Measurement System, a questionnaire was developed to ^ 
survey th§ school personnel perceptions and interpretations of the 
reporting system. The responses to the questionnaire were ^examined 

for all 22 test bpoklets simultaneously (see Appendix A for a summary 

\ 

o % f the evaluation data) * 

I. Recipients of Results 

An evaluation questionnaire was sent to 524 test administrators 
on 136 campuses in 84 school districts which field tested the 
career education instruments. Approximately 29% of the 524 
* . persons contacted returned useable evaluation forms . Of those 
.who did respond, 93% said they had received the results of their 
career education tests; these included 63% teachers, 28^ 
counselors, and 9% other school personnel. / . , 

II. Reporting Format . f 

These respondents were asked to'^rank (according to their pre- 
ferences) five different types of reporting* formats . The formats 
that were ranked are : , . • 

1 >' 

A Outcome Mastery - whether or not the student h#s mastered 

/ ' 

the outcome specified / j 



B. Level of Outcome Mastery-.- whether or not the student 

has mastered a degree of an outcoiye* 

' \ ; * ) ' ' 

C. , Areas of Proficiency - whether or not the student has 



demonstrated proficiency in one or more of several areas 



D. Item Response - item information which gives actual stri- 
dent response to a given test question 

E. School Curriculum Objective - Referenced Evaluation 
„ % (SCORE) - individual mastery or r>on-mastery (+,-,%) 

and class performance (%) £or one or ,several student 
outcomes . 

Format E is used by the contractor and it is common among other' 
•commercial test ?corers. Formats A through D were used as needed 
by the type of test item, and all four of these formats may have 
appeared on one report. 

. •■ , t> • • .. . 

Below is a chart shpwing the respondents 1st through 5th choices 
for formats A through E. The SCORE format (E) was' most often 
preferred while the outcome mastery \(A) format was least often 
preferred. The Item Response and Areas of Proficiency formats / 
were given as 2nd and 3rd choices. No discernible difference 
could be found between the SCORE ' (E ) ahd Level of Outcome Mastery 
(B) formats due to incomplete information on the respondents 
evaluation forms.' Only. 18 'of the" 151 respondents are reported' to. 
have seen all five type's of reporting formats. It is not Known 
how many have been exposed to formats other than the SCORE format. 



Format Most 



Type of Format 



D. 



Helpful in 
Elanning 



Outcome 

_Mas_tery 



Level of 
Outcome . 
Mastery 



-Are as of 



Item 



SCORE 



P r of i"c i ency . __Resj>on s-e 



N 



1st Choice 
2nd Choice 
3rd Choice 
4th Choice 
5th »■ Choice 



7 ( 4.6) 
13 ( 8.5) 
20 (13.1) 
28. (18.3) 
©(40.5) 



No Response; 23 (15.0) 



Total 



153 (100%) 



T 



N 



% 



14 (9.1) 

27 (17.5) 

34 (22.1) 

(45) (29.2) 

11 ( 7.1) 



32 (20.8) 

(Zj) (37.0) 

32 (20.8) 

10 ("6.50 

3 ( 1.9) 



22 (14. 3) 5 



20 (13.0) 



N 



N 



(37) (24.5) 

20 (13.2) 

(33) (21.9) 

.19 (12.6) 

^9) (19,2) 



(53) (34.9) 

21 (13.8) 

10 (6.6) 

26 (17.1) 

24 (15'. 8) 



13 (8.6) 



18 (11.8) 



154 (1007.) 



154 (100%) 



151 (100%) 



152 (100%) 



^C. " Reading the Report" . 

An interpretive guide, "Reading the Report", was sent along 
with the test results. CSlightly different guides were pro-* 
.vided for both the Level of Outcome Mastery and SCORE reporting 
formats.) Approximately 73% of the respondents said that the 
• guide was helpful to them in understanding the test results. 
IV. Understanding the Test Results 

The respondents were asked whether they' thought the test results 
were mean ingful, and use ful, readable and clear, and' provided 
too much or the wrong kind of information. 



„ Approximately 

59% said the results were meaningful and useful, 
. * 66% said the results were readable "and clear, 



10% said Lhe results ' g ave them too much iiifo rotation, and 



. 51% saf3 the, results were the wrong kind ot information. 
There were no indicators as to what was meant by "wrong kind of 
.information." , " v 

V. Audiences for Test Results 

The respondents were also asked who should receive a copy of 
the test results. j Teachers, curriculum consultants, adminis- 
trators, and students respectively were mentioned as the desired 
audiences for the test results. 
VI\. "Timing" of Returning Test Results 

"When should the test results be received" is a crucial 
question. In answer to this question, approximately 77% of 
the respondents said they preferred to receive the test results 
% at the beginning, rather than the middle or end, of the school 
W yearW 
VII. Testing Time Available 

The respondents were asked^to indicate how many career educa- 
tion tests they could administer to their* students . Approxi- 



/ mately ^ 4 

. - 7^% said they wojild be willing, to use 0 -to 5 tests, 
• t . 11% said they would be willing to use anywhere from 

6 to 22 tests, and 4 
i . 11% said that they would not use any of the tests. 

4 It is obvious that the schools are willing to give tests in only 



a few areas of career education,, wherever the need ia .the greatest 

V " * . v , 

:RiC ' 6-8 



! 

f 



VIII. Test Administrators 

In general, the respondents indicated that^either * the 
teachers or-counse Lor s should be rhe^nriRS r.o administer: 
"the career "education tests in tTieir schools. 



Possible Next Steps 

1/ The measurement system be examined so that the "right kin.ds 

^ of information" will be reported to schools. 

* « * 

\ 2. The reporting system be revised to adhere more closely to 

the SCORE format. 

3. Plans be developed for "fall testing", with a short turn- 
around time for reporting purposes. 

4. A survey or screening test be developed so that students can 

be tested in areas of greatest need. 

\ „ , * f.-. 

5. Whenever' possible, the' teacher or counselor be the one to 

administer the career education tests in, their schools. 

* * ** 

6 Teachers, curriculum consultants, administrators, and students 

.be involved in any post-test workshops developed for career 

* * 

education. 



9 

ERIC 



9 

7 



/ 



* Appendix A 

Evaluation Form for 
Career Education Repbrting System 



ERIC 



10 




. . TEXAS EDUCATION AGENCY \ 

DISTRICT NAME ' Division of Program_Planning CO./DI'ST. NO. 

and Needs ^Assessment 

COUNTY NAME > ~ TEST BOOKLET NO. 

— Evaluation Form for. . 

Career~Education Reporting System 

POSITION/ TITLE- -~ -•— : , 

■ '.YES NO . ' ' 

> . I. I have reviewed the results of the cjareer education test I 

administered. (If no, do not complete the form.) 

II. Which of the following formats for reporting the results of 
<^the career education .test would be most helpful for planning 
instruction? Assign a "1" to the format you favor most, a 
"2" to the next and continue through "5". 

A, The student has mastered the^outcome specified. 
• * S^ es ^° 

B. Thl student has .* The student has 1 The student did 

mastered the * some mastery of not master the 

outcome the outcome outcome 

* ** 1 *** 

, C. The student has demonstrated proficiency in the following 

areas : j 
•\ 1. . . . \ . 

2 . 

3 

* • eft ' ' 

] D. Item information which gives actual student response to 
"~ 1 : test qufe^tions. Example: The student feels that ,! My 

i ' social studies assignment is usually": 
t * ' * 1. very enjoyable - 

' 2 . enjoyable 

* X 3. alright but a chore 

4. boring- : 
5/ a waste of time * - 0 

E. OUTCOME 12 3 4 5 STUDENT PERFORMANCE # 

~ 1 A$Le, Ron + + 40% 

. * Adams, Cindy + + + t 60% 

Cadena, John + + . 40% 

■ Zeigler, Bill + + +' 60% 

i*i .* 

Y jjffi'**' * class t 

' • performance 75% 25% 50% 50% ! .50% 

RETURN T6: ' 
Keith Cruse ** t * 

Division of Program Planning 
.. and Needs Asses sweat PLAN - 02.9 

Texas Education Agencyv. 11'* 
O 1 East 11th Street * . > 
ERjC stin, Texas 78701 9 
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Does the interpretive guide "Reading the Report 11 help 
you understand the test results? 

* Are the results of the Career Education Test that were 
reported to you: 



meaningful alrhL useful 

*' B.* readable and clear • -.— 

C. too much- information 

D. wrong kind of information 

Who shouLd receive a copy of the results (may choose 
more than one)? 

teachers 

curriculum consultants 

campus administrators 

m r district administrators 



service center personnel y ~ 

students' 

; parents 

community 

Whentfould-you n^ed a report of the test results in orcler 
to be useful in the classroom?* & 

beginning of the school year \ 

middle of the school year'* \ 

end of the school year 



If your students were 'found to be weak iA several areas of 
career education, how many such tests would you' be willing 
to administer during one school year? (Check below the one 
you favor.) ' 

0 to 5 6 to 10 11 to 15 ' 15 to 22 NONE 



Who should administer the test? <t Check the appropriate one(s) 

teacher 1 

counselor 4 

other 

(specify) . - 



If you have any additional comments or suggestions to improve 
the reporting system or other aspects of the Career Education 
Measurement System, please use the remainder of t the form. 



COMMENTS: 
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